Season Data Quality Summary

Josh Dillon, Last Revised January 2022

This notebook parses information from each nightly rtp_summary notebook (as saved to .csvs) and builds a variety of useful tables. It is designed to highlight general trends across days.

Load Data

Figure 1: (Unflagged) Antennas in the Array

This plot shows the progression of the number of antennas in the data and the number of antenna-days that are not automatically flagged.

Table 1: Per Night Flagging

This table shows how many antennas we in the data for each night, the effective number of unflagged antenna-days, and the fraction of antennda-days flagged by:

Note that an antenna can be flagged by both auto_metrics and ant_metrics, so those quantities may add up to more than 100%. However an antenna flagged by either auto_metrics (for the night) or ant_metrics (for the file) will be excluded from redcal, so these percentages will tend to be small. Dashes indicate missing metric data, likely because part of the RTP failed to run. Also included are links to individual rtp_summary notebooks.

Table 2: Per-Antenna, Per-Night Flagging

This table summarizes information about individual antennas for the whole season and in the last seven days. This includes the number of days the antenna was in the data, what fraction of them were flagged, and how many unflagged days that leaves. It also attempts to figure out which metric led the most flagging, by seeing which step produced the most flags and when which median metric was largest within that step. Finally, it includes per-night flagging fractions.

Table 3: Per-Node, Per-Night Flagging

This table summarizes flagging by node, showing how many antennas are in the data from each node, how many antenna-days were observed in each node, and how many of them were unflagged (both for the whole season and the last 7 nights).